Session 11: Prosody

This paper provides a brief introduction to prosody research in the context of human-computer communication and an overview of the contributions of the papers in the session. In large part, prosody is "the relative temporal groupings of words and the relative prominence of certain syllables within these groupings" (Price and Hirsehherg [1]). This organization of the words, as Silverman points out [2], "annotates the information structure and discourse role of the text, and indicates to the listener how the speaker believes the content relates to the ...prior knowledge within the discourse context." For example, the relative groupings of words can provide cues to syntactic structure as well as discourse segmentation, and the relative prominence of words can provide cues to semantically important or focused items. Segmentation and focus represent two of the major uses of prosody, but other information may also be cued by intonation patterns, e.g. indication of continuation, finality or a yes-no question with phrase final "boundary tones". Prosody is typically also defined with a reference to its suprasegmental nature: "Prosody comprises all the sound attributes of a spoken utterance that are not a property of the individual phones" (Collier) [2]. In addition, prosody can operate at multiple levels (e.g., word, phrase, sentence, paragraph), making computational modeling of prosody particularly challenging. The acoustic correlates of prosody, which include duration of segments and pauses, fundamental frequency (F0), amplitude and vowel quality, may be influenced by prosodic patterns at more than one level, as well as inherent seg-mental properties. Modeling the interactions among the different factors is an important and difficult problem. Most current linguistic theories of prosody include an abstract or phonological representation of prosody to characterize aspects of phrasing, prominence, and intonation or melody. However, here we also see that abstract representations are of interest for computational modeling. Since it is generally agreed that prosody is not directly related to standard representations of syntactic structure , it is useful to have an intermediate representation to facilitate automatic learning and to simplify model structure. Thus, the form of an abstract representation is an important issue. Ideally, it should include all three main aspects of prosody, and address the needs of theory and computational models. Many different schemes have been proposed, and variations of two different prosodic transcription systems are used in the papers presented in this session. The TOBI (Tones and Break Indices) system for American English [3] is …